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The integration testing and system testing strategies used in the 
development of the Traffic Service Position System No. IB (TSPS No. 
IB) are described in this article. It discusses novel methods developed 
for the integration and testing of the first embedded application of 
the 3B20D Processor and DMERT operating system. The new testing 
techniques and project management aides, which were vital to the 
achievement of system quality for the Bell System service, are de- 
scribed. The first TSPS No. IB went into service on November 20, 
1981, in Fresno, California. 

I. INTRODUCTION 

The first Traffic Service Position System No. IB (TSPS No. IB) 
was placed in service in Fresno, California, on November 20, 1981. This 
newly installed TSPS brought modern Stored Program Control (SPC) 
operator services capabilities to telephone customers in Fresno and 
other San Joaquin Valley communities, and permitted Pacific Tele- 
phone to close the largest remaining Bell System Cordboard installa- 
tion in the United States. Modern operator services 1,2 are bringing 
convenience to telephone customers and increased efficiency and sav- 
ings to Pacific Telephone. 

Placing the Fresno TSPS No. IB in service marked the conclusion 
of a major testing program that was vital to the delivery of a high- 
quality system, capable of meeting all objectives. 1 This was especially 
critical because of the planned, rapid conversion of existing TSPS 
No. 1 sites to TSPS No. IB. Furthermore, since TSPS No. IB is the 
first electronic Stored Program Control system to utilize an embedded 
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3B20 Duplex (3B20D) Processor and its Duplex Multi-Environment 
Real-Time (DMERT) operating system, 3 confirmation of its capabili- 
ties was essential to the success of the TSPS No. IB program, as well 
as to other systems utilizing the 3B20D Processor. 

Following the successful introduction of TSPS No. IB in Fresno, a 
second new start TSPS No. IB was placed in service in San Antonio 
(Southwestern Bell Telephone) on January 30, 1982. Both the San 
Antonio and Fresno sites were instrumental in the overall test program. 
The first conversion of an existing in-service TSPS No. 1 office to 
TSPS No. IB occurred at Redwood City, California, on March 13, 
1982. 

The balance of this paper will focus on the overall test program for 
TSPS No. IB, specifically on integration and system testing. These 
systemwide test activities are major components of the methodology 
used for development of TSPS No. IB (see Fig. 1*). Integration testing 
specifically includes: the introduction of various system components 
(hardware and software) into the development laboratories; the inte- 
gration of these components to form a stable operating environment; 
and testing to ensure that a sufficient set of capabilities exists to 
warrant full, comprehensive, and broad testing. System testing includes 
the later testing and is done to verify the system, as a complete 
functional product, before its release to the field. 

The integration and system testing of TSPS No. IB and its compo- 
nents—the 3B20D Processor, the TSPS Peripheral System Interface 
(PSI), the DMERT operating system, and TSPS application soft- 
ware — was a complex effort. Because TSPS was the first user of the 
3B20D as an embedded processor, a substantial cooperative effort with 
the 3B20D development team was scheduled to identify and solve 
processor and operating system problems that were encountered dur- 
ing the design, integration, and testing of TSPS No. IB. A tightly 
scheduled and controlled series of integration tests were performed by 
a team at the Bell Laboratories Indian Hill Laboratory in Naperville, 
Illinois, as new hardware and software capabilities were delivered to 
the system laboratories. After these new capability packages were 
integrated and certified as ready for system test, they were delivered 
to the system test teams at Indian Hill, Fresno, and San Antonio. The 
use of the two field sites provided settings similar to live TSPS offices 
for early identification of site-dependent problems and for verification 
of future in-service retrofit procedures with Western Electric engineers. 

The system test plan consisted of over 16,000 unique tests of three 
general types: 



* Acronyms and abbreviations used in this paper are defined at the back of this 
Journal. 
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Pig. i — Test schedule for system integration. 

(i) Functional tests— to verify that design requirements were met 

by new features. 

(«') Regression tests— to verify continued proper operation of pre- 
viously delivered features. 

(Hi) Stress tests— to parametrically determine and evaluate system 
limits, response characteristics, and overall performance under traffic 
overload conditions and fault conditions. 

As depicted in Fig. 2, system test responsibilities were allocated to 
teams at the three locations with some intended overlap. For example, 
most of the DMERT feature system testing was scheduled for Fresno, 
to take advantage of the higher simulated-traffic-load capability at 
that site. Of these tests, some had been previously run at Indian Hill 
and at San Antonio, where exposure to different hardware and system 
configurations was critical. San Antonio was selected as the site to 
perform exhaustive tests of the existing TSPS periphery and to test 
external interfaces, such as to the Switching Control Center System 
(SCCS). In addition, a core of tests was periodically selected for all 
three system test sites to evaluate key software releases and to regu- 
larly assess the performance of a broad spectrum of features. In 
general, these "Run-for-Record" tests were a subset of the tests in- 
cluded in the system test plan. 

Both integration and system testing were pursued for a period of 
over one year to completely verify the operation of the TSPS No. IB. 

INTEGRATION AND SYSTEM TESTING 887 



Oil. 

a. cc 



o: ul uj z 



Li- — LU *- 






1-mOC 

Zylt 

— Q. UJ 



— Q. C/J 







888 THE BELL SYSTEM TECHNICAL JOURNAL, MARCH 1983 



Deliverables were phased to provide a smooth work flow. For this 
rigorous and demanding test program, tests were conducted ranging 
from microscopic tests, aimed at verifying operational status of specific 
elements of system capabilities, to global capacity and performance 
measurements and evaluations. During the test period, all problems 
found were tracked and pursued to their resolution. Performance 
criteria were set, performance was measured to ensure convergence to 
these criteria, and strong project management techniques were em- 
ployed to ensure a timely introduction of a high-quality TSPS No. IB. 
The following sections of this article describe how the laboratory 
integration and site system testing were conducted. The excellent 
performance of the TSPS No. IB system at cutover resulted not only 
from good planning and design but from timely integration and com- 
prehensive testing supported by appropriate documentation, develop- 
ment tools, and change control procedures. 

II. INTEGRATION TESTING 
2.1 Overview 

A crucial stage of the verification and evaluation process for new 
3B20D/DMERT and TSPS capabilities is integration testing. During 
the early stages of the process, the development environment was 
considered to be "non-frozen," and designers could freely make large- 
scale changes to the existing capabilities or introduce new ones. The 
design and development of various features were accomplished by 
partitioning them into a series of functional units, which were then 
tested by the designers to make sure that each unit met the design 
requirements. Upon the completion of unit testing by the designers, a 
set of integration tests were performed by an independent team to 
ensure that each functional unit performed reliably in the total system 
environment. Upon successful integration of functional units, they 
were considered "frozen," and changes were made only through formal 
procedures. The frozen functional units were then ready for system 
testing and evaluation, the last stage of the verification process. 4 

Certain unique characteristics of the TSPS No. IB development 
environment required special attention during the integration process. 
These characteristics can be summarized as follows: 

(i) TSPS No. IB was developed in a multilanguage environment. 
DMERT software was primarily developed in the high-level C lan- 
guage, while TSPS software was developed in both assembly language 
(TSPS emulated software) and the high-level C language (TSPS native 
software). 

(ii) DMERT software changes were concurrently developed by the 
common system organization and had to be integrated with the TSPS 
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software changes developed by the application organization. In some 
cases a DMERT change required a coordinated change from TSPS or 
vice versa. 

(Hi) 3B20D firmware/hardware changes had to be similarly inte- 
grated with the TSPS firmware, hardware, and software changes, such 
as those associated with the PSI. 

(iv) A new generation of TSPS No. IB support tools was developed, 
resulting in new load generation, installation, and integration proce- 
dures. 

The above characteristics were taken into consideration to establish 
an efficient integration and test strategy, which ensured the quality of 
the functional units integrated into the system. 

2.2 Integration testing methodology 

The integration testing philosophy adopted for TSPS No. IB estab- 
lished a rigorous methodology with systematic checks to independently 
evaluate all functional units supplied by designers in a single environ- 
ment. To meet this objective, it was decided that the integration group 
should be independent of the development groups. Independence 
provided an unbiased and fresh viewpoint in assessing the functional 
units and interpreting the test results. 

The integration test plan started early in the development cycle, 
subsequent to the availability of feature requirements. The plan took 
into consideration the availability sequence of various features and 
functional units within each feature. A strategy was established that 
guaranteed efficient handling of all units ready for integration testing 
without interrupting the other development activities. Special efforts 
were required to coordinate and synchronize the integration testing of 
TSPS functional units with 3B20D/DMERT features and capabilities. 
This was especially crucial in interface areas where direct interaction 
between the two environments existed. Another major consideration 
in the plan was that the software, firmware, and hardware environ- 
ments evolve in a manner compatible with the system test plan, thus 
allowing the system testing activities to proceed without any interrup- 
tion. 

The TSPS No. IB integration tests were generated independently, 
following a thorough analysis and review process, which is summarized 
as follows: 

(i) Feature requirement and design specification documents were 
reviewed to identify functional units within each feature. 

(ii) Functions performed by each unit, along with its input/output 
characteristics, were identified. 

(Hi) Software interfaces with firmware and hardware were exam- 
ined and reviewed. 
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(iv) 3B20D/DMERT interfaces with the TSPS application were 
thoroughly analyzed, and expected input/output responses of the 
interface units were identified. 

(v) The control flow among various units and their time sequence 
description was carefully inspected. 

(vi) A general layout and description of data used by each unit 
were identified. 

Based on these examinations, a set of integration tests were developed 
to exercise each functional unit within the total system environment 
and to stress 3B20D/TSPS interfaces. These tests were later aug- 
mented by the information supplied by the designer (s), upon comple- 
tion of unit testing, to start the integration testing process. During this 
process it was first verified that each unit correctly performed all 
intended functions, including its interaction with other units in a single 
environment. The second, more subtle, objective was to ensure that 
the unit did not perform any function that, either singly or in combi- 
nation with others, would degrade the performance of the system. 
Third, each unit was examined to make certain that it met the 
standards set for design structure, documentation, and coding. 

Problems identified during the process of integration testing were 
classified according to their impact on the overall system. Problems 
were prioritized and fed back to the designer(s) for corrective actions. 
Problems related to the TSPS software were primarily tracked by 
Trouble Reports (TRs), while Modification Requests (MRs) were used 
to track 3B20D/DMERT problems. Specific corrective action by a 
designer was required to close a TR or an MR. 5 The list of all open 
TRs and MRs was carefully monitored to evaluate the total impact 
and to establish a plan for closing each individual TR or MR. The 
primary objective of this plan was to ensure that the evolution of the 
TSPS No. IB software, firmware, and hardware could proceed on 
schedule without any interruption. 

2.2.1 DMERT integration testing 

The development of DMERT software was done in parallel with the 
development of TSPS software. At well-defined points of the devel- 
opment, a full DMERT release was generated by the common system 
organization and delivered to the various applications after being 
tested on a stand-alone basis. 6 

Upon delivery, each DMERT release was first installed in the 
Program Support System (PSS). Subsequently, steps were taken to 
incorporate the new DMERT release into the TSPS No. IB environ- 
ment (see Fig. 3). These steps can be summarized as follows: 

(i) The Equipment Configuration Database and System Genera- 
tion Database (ECD/SG) were updated to reflect the latest DMERT 
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changes. These databases contain data structures necessary to gener- 
ate and run the TSPS No. IB software system. 

(ii) Those areas of TSPS software that were directly affected by 
the new DMERT release were identified. The corresponding TSPS 
software changes were then developed on the PSS under the control 
of the Change Management System (CMS). More information about 
the software generation process may be found in Ref. 5. 

(Hi) The TSPS official native-mode software was recompiled using 
the new DMERT release to update the TSPS programs that were 
affected by the DMERT release (such as a library that is shared by 
both DMERT and TSPS software and is changed by the new DMERT 
release). All TSPS programs that were changed by the recompilation 
process were audited for potential impact on the overall system. If 
necessary, appropriate TSPS software changes were generated [see 
Step (ii)] to compensate for the impact. 

(iv) Those areas of DMERT that required changes unique to the 
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TSPS application were identified. These changes were also reflected 
in the PSS. 

Subsequent to these steps, a full TSPS No. IB load was generated on 
the PSS and was installed in the system laboratory for load recovery 
and integration testing. This load contained the latest DMERT release, 
along with changes unique to the TSPS application; a recompiled 
version of the TSPS official load; ECD/SG database changes; and the 
TSPS software changes required by the new DMERT release. 

Any problems in the new load were quickly identified using the 
debugging tools associated with the TSPS No. IB, 5 and immediate 
steps were taken to obtain the necessary changes from the appropriate 
designers (DMERT or TSPS). Once the load was cycling, a set of 
integration tests were conducted to ensure that each functional unit 
performed as expected in the total system environment. These tests 
were based upon the ones independently generated by the TSPS 
application organization, augmented by those supplied by the 3B20D 
common system organization. Special emphasis was given to DMERT/ 
TSPS interface modules, TSPS areas directly impacted by the new 
DMERT release, DMERT changes unique to the TSPS application, 
and ECD/SG database changes. Problems identified during the proc- 
ess of integration testing were tracked using the strategy described in 
Section 2.2. 

2.2.2 TSPS integration testing 

Modifications to the TSPS software were introduced into the TSPS 
No. IB environment via base loads. 5 Introduction of a new base load 
began with identification of all source file changes that had occurred 
since the prior base load. Then a new test load was created, containing 
additional software changes and associated ECD/SG database 
changes. New software changes were usually grouped in one or more 
test packages. The packaging strategy for the TSPS changes took into 
consideration many factors including: 

(i) Keeping a high degree of resolution in testing of various 
changes that have an impact on the same programs. 

(ii) Optimizing the amount of time used in the system laboratory. 

(Hi) Keeping the PSS effort for compiling various changes to a 
reasonable amount. 

(iu) Optimizing the system capacity for processing software 
changes. 

Keeping these objectives in mind, all software changes were first 
mapped into a series of test packages. Program dependencies played 
a key role in determining the number and content of these test 
packages. The load of the test packages, along with coordinated 
hardware and firmware changes, were then installed in the system 
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laboratory separate from the TSPS official load before the integration 
testing could proceed. 

Upon installation of the load, a predefined set of integration tests 
were used to examine each test package. These tests were aimed at 
quickly identifying problems and attributing them to individual 
changes. Problems encountered during the testing were classified by 
their impact on the normal operation of the system and tracked as 
described in Section 2.2. This process was then repeated for all test 
packages until the necessary resolution was obtained, and it was 
ensured that all changes could coexist and perform as expected in a 
single environment. All changes that successfully passed the integra- 
tion testing were then installed in the system laboratory as the new 
official base load (see Fig. 4). 

Upon completion of the integration testing, functional units were 
considered "frozen" and ready for system testing. In the frozen envi- 
ronment a set of stringent change control procedures were used so that 
the TSPS software evolved in a rigorously controlled manner, leading 
to a high-quality production release. All problems were documented 
and tracked by appropriate trouble reports, which were carefully 
monitored. To close a trouble report, a designer was required to submit 
a Correction Report (CR), which contained the functional description 
of the change, all necessary information relevant to the software 
change, and a description of the unit tests used by the designer. Each 
change was considered an independent entity and was individually 
tested in the total system environment before its approval. All changes 
had to be approved by the Change Review Committee (CRC) before 
they were incorporated in the official load. The CRC comprised 
representatives from each hardware/software design and test organi- 
zation on the project. 

2.3 Testing of 3B20D and TSPS firmware and hardware changes 

Firmware and hardware changes associated with the 3B20D were 
also tested in the system laboratory environment by the TSPS inte- 
gration group to uncover problems unique to the TSPS application. 
These changes, along with coordinated software modifications, were 
first installed in the system laboratory environment. A series of inte- 
gration tests were then performed to stress and exercise the firmware- 
hardware-software interfaces. Problems encountered during this stage 
of testing were tracked in a fashion similar to that described in Section 
2.2. 

In cases where 3B20D microcode changes had an impact on the 
writable portion of the microstore, 7 no firmware change was necessary, 
and the microcode change was released in a fashion similar to a 
software change described in earlier sections. 
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Changes to the TSPS microcode were administered under control of 
CMS in a similar fashion to the TSPS software changes. A change was 
first developed on the PSS and was then submitted for integration 
testing. The change was transferred to the system laboratory in one or 
more files and was installed in the writable microstore unique to the 
system labs. A series of integration tests were then performed to verify 
its integrity in the total system environment. Similarly, all hardware 
changes associated with TSPS were tested in the system laboratory 
environment using prototype circuits prior to their official availability. 
Once it was verified that the hardware/firmware changes operated as 
expected, they were transmitted to Western Electric along with man- 
ufacturing and factory test information, and were subsequently in- 
stalled at the field sites. 

2.4 Distribution of individual software changes 

Once a stable TSPS No. IB release was installed in a field site, the 
subsequent individual software changes were supplied as individual 
packages and were installed using the field update commands. 5 A 
package typically consisted of the software change, additional files 
containing field update commands, and information files required for 
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installation, testing, soaking, and distribution of the change to the 
field. 

For a change in the 3B20D/DMERT software, the 3B20D common 
system organization generated a software package that was distributed 
to each application organization. Before the release of the package to 
the various applications, it was first tested and certified in the common 
system environment by the common system organization. Once the 
change was received by TSPS, it was examined for compatibility with 
the TSPS environment. In some cases a coordinated TSPS software 
change was required before actual installation and testing could pro- 
ceed. A series of tests were then conducted to ensure that the change 
performed as expected within the total system environment. These 
tests included those supplied by the common system organization, as 
well as an independent set generated by TSPS personnel. 

Once a software package was successfully tested and soaked in the 
TSPS system laboratory environment, necessary changes were made 
to its information files prior to field delivery. These changes were made 
to delete the test information unique to the system laboratory envi- 
ronment. This information was used by application organizations to 
verify the modification and was not applicable to the field environment. 

The TSPS application software changes required for distribution to 
the field sites were individually tested and approved using the "frozen 
environment" methodology described in Section 2.2.2. These changes 
were then packaged using the standard format described earlier in this 
section. Necessary considerations were given in the packaging strategy 
to balancing and optimizing various factors such as the field installation 
time and the number of system initializations required to install the 
changes. Also, if necessary, these changes were retested to ensure that 
no problems were introduced in the final packaging. 

III. SYSTEM TESTING AND EVALUATION 
3.1 Overview 

System testing also represented a crucial and essential part of the 
overall verification process. During this stage a complete set of func- 
tional tests were written and executed to independently evaluate all 
TSPS and DMERT operational and maintenance capabilities from a 
system viewpoint. These tests were systematically performed with the 
objective of ensuring that the requirements for each feature were met. 
Special emphasis was given to evaluating the interaction between the 
new features and the existing features that were emulated from the 
TSPS No. 1 environment. Furthermore, regression tests were per- 
formed to ensure that previous tested capabilities were not being 
adversely affected by the introduction of new ones. All problems 
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identified during the system testing interval were documented by 
appropriate trouble reports. 

The system testing effort was divided into several areas with differ- 
ent objectives. They are listed below with the primary objective for 
each area and are further discussed in Sections 3.2 to 3.5. 

(i) Call-processing objective — Verification of all emulated call- 
processing features. 

(w) TSPS maintenance software objective — Verification of fault 
recognition, routine exercises, and diagnostics. 

(Hi) External interfaces objective — Verification of external inter- 
faces such as the Service Evaluation System (SES), Switching Control 
Center System (SCCS), and Billing Validation Application (BVA), etc. 
(see Refs. 1, 8, 9, and 10). 

(iv) DMERT and 3B20D system testing objective— Verification of 
DMERT and 3B20D features, DMERT and 3B20D/TSPS interfaces, 
and sample faulting of 3B20D and its periphery in the TSPS environ- 
ment. 

(v) Regression testing objective — Determination of impact of new 
TSPS and DMERT/3B20D releases on previously tested capabilities. 

(vi) System evaluation runs objective — Analysis of overall system 
stability and quality at various loads. 

System tests were performed both in the system laboratory and the 
field sites. The field test plan consisted of functional tests, environ- 
mental tests, and an overall acceptance test, and was used as a final 
check that the system met its functional objectives at specified envi- 
ronmental limits. Over 16,000 individual functional system tests were 
written and executed during the system test interval at the San 
Antonio and Fresno test sites. The first execution of all system tests 
was scheduled and accomplished by early in the third quarter of 1981 
(see Fig. 5). This timely execution enabled identification of problems, 
management, resolution, and closure by turnover (see Fig. 6). All 
significant system tests passed prior to the turnover of the Fresno site 
to the operating telephone company. The environmental tests verified 
system performance at the limits of high temperature and low voltage. 
The functional-site testing effort validated system performance in a 
fully equipped TSPS office that has been engineered by an operating 
telephone company with hardware supplied and installed by Western 
Electric. 

3.2 Call-processing testing 

The call-processing software in TSPS consists of a set of programs 
that provide the logic and control for processing telephone calls. These 
programs supervise the state of the call, transmit and receive signals 
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to and from other switching systems, send information to and receive 
signals from operators, and record billing details on the calls. 

In most cases the call-processing programs were directly emulated 
from the TSPS No. 1 environment without any structural or code 
changes. Because the call-processing software was preserved in the 
TSPS No. IB, the strategy for testing was mainly to verify the proper 
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emulation of the existing capabilities. This objective was accomplished 
by using system test facilities to generate simulated call inputs to 
exercise various call-processing capabilities, including those involving 
customers and operators. These capabilities were tested both on a 
single-call basis and under call loads simulating the normal traffic 
environment of the system. 

3.3 TSPS peripheral fault recognition, diagnostics, and exercise 

TSPS No. 1 peripheral maintenance programs such as fault recovery, 
diagnostic, and exercises were also preserved in the TSPS No. IB 
environment. These maintenance programs were emulated from the 
TSPS No. 1 environment, after necessary code and structural changes 
were made to provide interfacing compatibility with 3B20D hardware, 
firmware, and maintenance software capabilities. The function of fault 
recognition is to analyze failure conditions and quickly reconfigure the 
system to a working state with a minimum interruption to operational 
call processing. Testing in this area verified that faults were success- 
fully detected and appropriate actions were taken. 

Diagnostics are used in TSPS to test the various equipment units in 
response to fault-recognition requests or manual requests. These ca- 
pabilities resolve hardware malfunctions by providing trouble-locating 
messages for use by maintenance craft to repair the faulty hardware. 
Testing in this area verified that the diagnostics properly detected the 
presence or absence of troubles and produced messages consistent with 
expected results without adversely affecting system operation. 

Specific routine exercises are used to periodically examine the cor- 
rect operation of the hardware and to detect various failures that may 
not show up in normal system operation. In general, exercises utilize 
the diagnostic capabilities described above to verify the condition of 
the equipment units on a periodic basis. Testing in this area demon- 
strated that the exercise routines operate in the expected manner. 

3.3.1 Test strategies 

Sample physical fault insertion, power faulting, and data faulting, 
combined with manual actions initiated at the Local Maintenance 
Position, were the major means of verifying the proper operation of 
system maintenance functions. These techniques were used to develop 
specific tailored test strategies for each peripheral unit. Initially, tests 
were conducted in the absence of call load to verify the proper 
response. Subsequently, maintenance capabilities were tested in the 
presence of varying call loads to ensure proper interaction of call 
handling and maintenance functions. 

Physical fault insertion was one of the primary methods of testing 
the operation of maintenance software and hardware. Physical fault 
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insertion verified proper cooperation of fault recognition, system re- 
configuration, and diagnostics. It further provided a test of the Trouble 
Locating Manual (TLM) used by the craft to identify faulty circuit 
packs. It was impractical to do exhaustive fault insertion, such as that 
utilized in TLM generation. However, it was necessary to develop 
sample faults to verify that the maintenance software performed within 
the existing requirements. 

Power faulting was the second major means of introducing hardware 
faults. Power faults were introduced selectively by blowing fuses or 
intentionally removing power to verify the ability of the system to 
detect the fault, take the appropriate equipment out of service, and 
produce the proper alarm and output messages to enable craft person- 
nel to restore the system. 

Data faulting was the third faulting technique used in testing main- 
tenance actions. Data faulting is the application of erroneous data to 
the system. It measures the sensitivity of the system, the adequacy of 
defensive program checks, and the efficiency of data consistency 
audits. This type of faulting was useful, for example, in testing response 
to transmission irregularities between the base and remote subsystems, 
such as the Remote Trunk Arrangement and Position Subsystem No. 

2. 11 

Various manual actions at the Local Maintenance Position were also 
used to verify the proper response of the system in areas such as 
reconfiguration, diagnostics, measurements, control and display capa- 
bilities, and exercises. These types of actions are usually taken by the 
craftspeople both on a routine basis and in emergency-action condi- 
tions. 

3.3.2 TSPS peripheral test sequence 

Communication between the SPC 1A and peripheral equipment is 
accomplished by the Central Pulse Distributor (CPD), Communica- 
tions Bus Translator (CBT), Master Scanner (MS), Universal Trunk 
Scanner (UTSC), and Universal Trunk Signal Distributor (UTSD). 
Maintenance testing verified that the emulated maintenance programs 
detect malfunctions in the above units and take appropriate recovery 
and diagnostic actions. 

The TSPS periphery consists of a number of functional entities or 
subsystems (see Table I) that are administered by duplicated control- 
lers that interface with the SPC bus system. Extensive tests were 
performed to verify the operation and maintenance of these functional 
units under control of the TSPS No. IB software. 

In TSPS, trunks and service circuits provide the interface between 
the TSPS and external systems. For maintenance purposes, access to 
these circuits is required to evaluate performance and localize or 
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Table I — TSPS peripheral subsystems 

Position and Trunk Link Network 

Position Subsystem No. 1 

Peripheral Control Link 

— Position Subsystem No. 2 

— Remote Trunk Arrangement 

Station Signaling and Announcement Subsystem 

Common Channel Interoffice Signaling Links 



isolate a trouble to a particular faulty unit. Comprehensive tests were 
required to ensure proper operation of these trunk and service circuit 
routines. Table II lists the TSPS trunks and service circuits for which 
maintenance routines were tested. 

3.4 TSPS external interface tests 

A number of systems external to TSPS provide operator adminis- 
trative functions, or interact with TSPS for services and maintenance. 
In general, these are self-contained systems, but rely on communica- 
tions with TSPS over data links and/or channels, thus requiring 
coordinated testing between the two systems. The communications 
generally rely on specific protocols or message types. Testing of exter- 
nal interfaces verified that the necessary protocols, message handling, 
and maintenance states were operational in the TSPS No. IB environ- 
ment. Some of the external interfaces include: 
(i) External administrative centers 
(ii) The Hotel Billing Information System (HOBIS) 8 

(Hi) The Service Evaluation System (SES) 9 

(iv) The Billing Validation Application (BVA) 1 
(v) The Switching Control Center System (SCCS). 10 
The interfaces in items (i) through (iv) above required only verification 
that TSPS No. IB had not introduced operational or maintenance 
problems. The SCCS interface [item (v) above] required substantive 
testing because of new interface hardware and software (see Ref. 12). 
This interface was functionally tested between the system laboratories 
of the SCCS and TSPS development organizations. Field-site testing 
verified these functional capabilities and stressed load-related features 
that could not be tested earlier. 

3.5 3B20D/DMERT system testing 

3B20D/DMERT testing evaluated 3B20D/DMERT hardware and 
software in the application environment. The 3B20D/DMERT tests 
concentrated on interfaces, resource utilization, and system response 
characteristics in the TSPS application environment. As such, over- 
load response, resource audits, and software fault tolerance were 
extensively tested. The combined craft/machine interface and 
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Table II — Tests of TSPS trunk, service circuit, and 
maintenance circuits 



Service Circuits 

— Dial-Pulse receivers 

— Multifrequency receivers 

— Multifrequency outpulsere 

—High-Impedance, Multifrequency receivers 

—Coin Control and Ringback circuits 

—Reorder Tone and Announcement circuits 

— Audible Tone trunks 

— Coin Detection and Announcement circuits — Type 1 

—Coin Detection and Announcement circuits — Type 2 

—Dual Tone Multifrequency Detection and Announcement circuits 

—Multifrequency Announcement and Detection circuits 

Trunk Test Panel 

AMA maintenance 

Recorded Announcement equipment 

Tone and Interrupter circuit 

Time of Day circuit 



DMERT administrative functions (field utilities and field update, for 
example) were also extensively exercised. 

3.5.1 Fault recovery, initialization, diagnostics, and overload 

The TSPS No. IB maintenance strategy is based on the 3B20D/ 
DMERT maintenance design augmented by additional capabilities 
unique to the TSPS application. These additions are mainly in areas 
where TSPS software directly interacts with the DMERT software, or 
when specific hardware interfaces are required for the TSPS applica- 
tion. 

Specific fault-recovery strategies were developed by TSPS for the 
PSI, the interface between the 3B20D and the TSPS periphery. 12 In 
addition, the TSPS Application Integrity Monitor (AIM) was devel- 
oped to directly interact with the DMERT system integrity monitor 
(SIM) to ensure the system integrity of TSPS No. IB. DMERT system 
initialization and overload features were augmented by TSPS to es- 
tablish an overall system initialization and overload strategy for the 
TSPS No. IB. 13 These areas were tested extensively to determine the 
proper system response. 

The 3B20D and PSI diagnostics, Trouble-Locating Procedures 
(TLP), and Routine Exerciser (REX) were tested by sample faulting 
techniques. Manual requests for removal/restoral were also used for 
verification of capabilities. 

The TSPS No. 1 maintenance craft interface was replaced in TSPS 
No. IB with a combined 3B20D Processor and TSPS maintenance 
craft interface. Testing in this area concentrated on combined craft 
response, software and hardware alarm interfaces, and an assessment 
of process priorities to provide sufficient terminal response time under 
load. 
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3.5.2 Administrative functions 

Testing in this area concentrated in the areas of 3B20D recent 
change/verify (for equipment and system configuration database 
changes), system update (for installation of a "point" issue of a current 
generic or installation of a subsequent generic), and field update (for 
installation of one or more broadcast warning messages). 
3.6 System evaluation runs 

3.6. 1 Purpose and definition 

One goal of system testing was to monitor a set of quantified 
performance criteria to measure overall system quality and to ensure 
that system design goals were met. Of particular importance for TSPS 
No. IB was the evaluation of system behavior at full-load call handling 
(maximum capacity). This was particularly important since a substan- 
tial number of TSPS No. 1 to TSPS No. IB live in-service retrofits 
were scheduled for 1982 for offices at or approaching real-time overload 
of the SPC 1A. Since the Fresno site had a large amount of peripheral 
equipment, sufficient microprocessor-controlled call simulators and 
operator simulators were installed on a temporary basis to permit 
stressing the new TSPS No. IB above the capacity design objective. 
This additional performance margin simulated variation among sites 
with respect to call mix and peakedness in the busy hour(s). 

In the first quarter of 1981, a series of weekly System Evaluation 
Runs (SERs) were begun at both Fresno and San Antonio to measure 
system performance in an environment approximating post-cutover 
conditions. Typically, these SERs were executed on weekends and 
were 24 to 48 hours long. During these SERs the following conditions 

were met: 

(i) Operating telephone company maintenance craft were assigned 
to monitor and maintain the office, performing all TSPS peripheral 
maintenance and trunk cutover testing simulating an in-service office, 
(u ) Bell Laboratories and Western Electric site staff had instructed 
the maintenance craft in the use of normal 3B20D Processor and SPC 
IB peripheral maintenance procedures. When craftspeople were trou- 
bleshooting equipment faults or potential design problems, normal 
field repair tools and techniques were used. 

(Hi) A simulated call load was applied to the system. At Fresno, the 
first quarter 1981 SER was conducted at a load equivalent to 25 
percent of the system call capacity. This was gradually increased so 
that by mid-1981, SERs were conducted above the call-capacity design 
objective of the TSPS No. IB. 

3.6.2 Measurements and objectives 

A team of designers, testers, and system analysts defined a set of 
eighteen measurements that were tracked as part of the SER each 
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175% 


160% 


0.04% 


<0.05% 





<1/10 days 





0.02/wk 



Table III — Fresno Service Acceptance Test 
Performance* (9/27/81-10/5/81) 

System Per- First-Site Turn- 
formance over Objective 

Call Load (Percent of SPC 1A) 
Mishandled Calls 
Plug-in Replacements 
System Initializations 

* Includes normal maintenance and PT + T trunk testing. 



week to summarize system stability and maintainability. For each 
measurement, a turnover objective was specified. If several results 
deviated widely from the objective, a high priority was placed on 
resolution of these problems. When the measurements in those areas 
approached the turnover objective, resources were redirected to im- 
proving other areas of system performance. Statistics were kept on the 
frequency and cause of initializations, interrupts, audits, and overloads. 
The duplex performance of the processor and its disk subsystem was 
tracked and the degree of automatic fault recovery and identification 
was evaluated. Hardware failure rates were closely monitored and 
compared with reliability models. All initial service objectives were 
achieved prior to cutover. Table III shows Fresno Service acceptance 
test performance versus key first-site turnover objectives. 

IV. SUMMARY 

The strength of the integration and system test plan for the suc- 
cessful field introduction of the TSPS No. IB involved the use of 
complementary techniques to set objectives and monitor progress. The 
comprehensive functional and regression testing, close tracking of 
correction and modification requests, and prompt integration and 
delivery of a diverse set of software changes were essential to the 
steady progress summarized by the periodically executed System Eval- 
uation Runs. 

An immediate benefit of this approach was the customer satisfaction 
expressed by Pacific Telephone and Southwestern Bell Telephone at 
turnover, cutover, and subsequent operation of their new TSPS No. 
IB systems. Furthermore, the implementation of the comprehensive 
test program described herein is responsible for the excellent perform- 
ance of the 37 offices now in service (35 of which were live in-service 
retrofits in high- traffic sites). Current Bell Laboratories and Western 
Electric efforts involve the continued coordination and support of the 
large number of TSPS No. IB live in-service retrofits planned for the 
next few years. 
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